Proceedings of the Rapid Screening Technologies , Deception Detection and Credibility Assessment Symposium January 2013

In this paper, we investigate how degree of, and temporal changes in, interactional synchrony can signal whether an interactant is truthful or deceptive. We propose an automated, data-driven and unobtrusive framework using visual cues. Our framework consists of face tracking, gesture detection, facial expression recognition and interactional synchrony estimation. This framework is able to automatically track gestures and expressions of both the target interviewee and the interviewer, extract normalized meaningful synchrony features and learn classification models for deception recognition. To validate these proposed synchrony features, extensive analyses have been conducted on a database of 242 video samples and show that these features reliably capture simultaneous synchrony. The relationship between synchrony and deception is shown to be complex.